5.5 Monitoring and evaluation

5.5.1 stakeholder accounts
5.5.2 community involvement to plan the evaluation
5.5.3 community-based environmental assessment
5.5.4 strengths, weaknesses, opportunities and limitations (SWOL) analysis

5.5.1 Stakeholder accounts

Stakeholder accounts — verbal presentations based on a set list of questions about key aspects of the conservation initiative — can provide good grassroots perspectives about the initiative's operations and achievements. Field-based staff and stakeholder representatives prepare a presentation for a meeting with the community and the management of the initiative. Visual materials may or may not be used, depending on resources available and the type of issues to be covered. A variety of groups should be given a chance to present to ensure that all the issues are covered in depth and that the interests of all stakeholders are considered. Each presentation should take no longer than 20 minutes.

Purpose

Verbal presentations are a form of story-telling. As such, they can be a natural and non-threatening way of communicating concerns and ideas for some traditional cultures. People listen to the accounts, assess the messages, ask questions to clarify particular points, and then in a group setting (involving both the local people and management of the initiative), decide what changes need to be made and how these can be achieved.

Steps in using the tool

• Together with the local stakeholder representatives (e.g. the Conservation Council if one exists) prepare a list of topics to cover in the presentations. Questions should be open-ended and may differ for field workers and stakeholders. Appropriate questions for stakeholders could be: "What has changed for you since the initiative began? What has changed in the environment? How do you know these changes have occurred? What do you think is working well in the initiative? What is not working so well? Have you had any problems with the initiative staff/management? Are you happy with the way these problems were dealt with? Can you suggest things that would make the initiative work better?" Some of these questions would also be appropriate for the field workers. The field staff may be interested in presenting on topics such as the adequacy of their training, supervision and provision of resources. These may be addressed but they are best dealt with in detail in a separate meeting for staff only.
  
• Identify which stakeholder representatives and field staff will give the presentations and provide each of them with the list of questions to use as a guide. Stress that they should feel free to raise other matters they feel need to be covered. Allow at least a week for people to prepare their thoughts before the meeting.
  
• At the evaluation meeting, encourage those present to ask questions after each presentation, and to add to the information presented.
  
• Note the main points/issues to be addressed on a blackboard/flip chart, etc.
  
• At the end of the presentations, review the main points, and discuss options/strategies for addressing the issues/problems raised and for building on the successes.
  
• Before coming to any conclusions, it may be appropriate to follow up the presentations with a field visit to view specific aspects raised in the presentations.

• provides an effective two-way discussion process where local people and staff participate on an equal basis;
  
• involving stakeholder representatives in setting the questions, or indicators, on which the success of the initiative will be evaluated, fosters a sense of ownership and responsibility for the outcome, and reinforces the message that they have a vested interest in the initiative;
  
• asking field workers to contribute presentations gives them reassurance that the management of the initiative values their opinions and concerns;
  
• verbal presentation format allows those who are illiterate to participate on an equal basis;
  
• unlike a written report, the process enables immediate clarification of information and feedback on the issues raised;

• some presentations can be highly subjective and biased;
  
• the process can be time-consuming (a full evaluation meeting could take an entire day, especially if a field visit is included);
  
• the quality of the facilitation is important to ensure that conflicting opinions are dealt with constructively and that all the issues raised are accurately recorded and considered to the satisfaction of the meeting.

Evaluating initiatives provides an opportunity for both outsiders and insiders to reflect on the past in order to make decisions about the future. In a participatory process to design an evaluation, insiders are encouraged and supported by outsiders to take responsibility for and control of planning what is to be evaluated and deciding how the evaluation will be done. Much of the material acquired from the participatory information gathering, assessment and planning exercises can be used in participatory evaluation.

Purpose

Involving the community in developing an evaluation process ensures that all aspects of concern are covered. It also enables the skills and knowledge available within the community to be identified and utilized for information collection and analysis. This reduces reliance on outsiders (e.g. consultants) who may be much more expensive and less informed to do this work.

The results of the evaluation exercise should enable decisions to be reached on whether to change the objectives of the conservation initiative, change the strategy, change activities or continue all or some of these. In a participatory evaluation both specific activities and the objectives of the initiative are considered, with the objective of learning what worked and why, and what was not successful and why it wasn't.

Steps in using the process

These steps can be undertaken with a group of stakeholder representatives (e.g. a Conservation Council) or in a meeting open to everyone.

• Review the objectives and activities of the initiative and the reasons for the evaluation i.e. "what do we want to know?"
  
• Develop evaluation questions — these can be written on large sheets of paper or a blackboard — and rank them.
  
• Decide who will do the evaluation (e.g. the whole community in an open meeting, a team representing major stakeholders, or an outside consultant).
  
• Identify direct and indirect indicators. Direct indicators are facts and information that directly relate to what is being measured (e.g. the number of cattle owned by a family). Indirect indicators provide information on aspects which cannot be easily or accurately measured (e.g. whether a family possesses a radio or a bicycle can, in some communities, be an appropriate indirect indicator of its total wealth).
  
• Identify the information sources for evaluation questions. If the information is not currently available, decide which information-gathering tool would be appropriate. If a tool has been used before, it may be used again to update the information and show the change that has occurred.
  
• Identify the skills and time required to obtain the information, including any expertise that may need to be recruited from elsewhere.
  
• Decide the time-frame for gathering and analyzing information. Timing needs to take into account factors such as seasonal constraints, religious holidays, and field staff availability.
  
• Decide which people will gather which information. If an outside evaluator is to be employed, designate someone to whom this person will report.
  
• Decide how the information should be analyzed and presented to the wider community and the staff of the conservation initiative for discussion and drawing conclusions.

• ensures that a broad range of issues are covered in the evaluation;
  
• ensures a more comprehensive and better designed evaluation plan;
  
• prevents information from the field being filtered by staff to reflect their own interests;
  
• involving stakeholders in the evaluation fosters a sense of ownership and responsibility for the outcomes at the local level;
  
• fosters the development of evaluation skills within the affected community;
  
• builds bridges and strengthens communication between the stakeholders and the conservation initiative;

• stakeholder interests can override conservation interests if there is not someone specifically representing the interests of the conservation initiative;
  
• the process of developing and participating in an evaluation can be time-consuming; this may limit the number of stakeholders willing or able to take part.

Community-based environmental assessment provides a community perspective on the state of the environment, prior to or during a conservation initiative, as part of a monitoring or evaluation exercise. A list of environmental aspects or factors is agreed upon by the community. The state of each factor is determined by allocating a certain value (e.g. excellent, good, poor, disastered, etc.) or number to it. It is not the actual value or number that is important but the way those change over time as recorded by ongoing observations.

Purpose

Community-based environmental assessment provides a framework by which insiders can make observations and judgements about the state of certain environmental factors.

Steps in using the process

• In a meeting with concerned community members, discuss the purpose of the assessment and how it can be carried out.
  
• Decide what is to be measured (e.g. well-being of the community, well-being of a particular natural area) and define what indicators will be used (e.g. abundance of specific species in the area, pollution, soil erosion, migration, morbidity and mortality, wealth, literacy, access to clean water, and so on).
  
• Write up the values to be used and what each represents
  
• g. 5 = very good; 1 = very bad).
  
• Draw up a list of all the items to be evaluated. If the group is small (less than ten) work through the list together to reach a consensus on what value should be attributed to each item at the present time. If the group is larger, divide into smaller groups, with each group having the same list of items to evaluate. Then bring the groups together to negotiate a common list of allocated values. Record and store the results and decide when the exercise will be repeated (e.g. after six months or one year).
  
• At the agreed time, repeat the exercise of assigning a value to the items to be assessed. Discuss the reasons for the values attributed and the causes of changes since the previous exercise (if relevant).
  
• Identify the actions which need to be taken in response to the analysis and who should take responsibility for each task.

• enhances local knowledge of environmental issues;
  
• creates an awareness of the potentially negative and positive environmental impacts of activities;
  
• fosters the development of evaluation skills among participants;

• this is quite a complicated tool and a clear explanation is required to make sure it is well-understood before assessment begins;
  
• some value allocations may be highly subjective, although discussing the reasons for the allocations can help reduce and clarify this;
  
• it may be difficult to reach consensus on "values" where there is significant disparities between the costs and benefits experienced by different stakeholders in relation to the relevant item.

Strengths, weaknesses, opportunities and limitations (SWOL) analysis is a structured brainstorming process to elicit group perceptions of a specific aspect of, for instance, a community, environment or project. The aspect is analyzed in terms of the positive factors (strengths), negative factors (weaknesses), possible improvements (opportunities) and constraints (limitations).

Purpose

SWOL analysis can be useful for evaluating activities carried out in a conservation initiative. It can be focused on specific aspects of the initiative, such as services provided by external agencies or activities being undertaken by a local community. It can also be used by specific interests (or stakeholders) to clarify their views on a proposal before meeting with other interest groups.

Steps in using the tool

• A number of specific aspects/topics to be evaluated are identified and listed one below the other on a blackboard or sheet of paper.
  
• A four-column matrix is drawn on the side of the first column, and the four categories are explained to participants. To this end it may be helpful to phrase the four categories as questions e.g.. "What are the good things about this particular service/activity, what has worked well?" (S); "What are the things that have not worked well? (W); "What chances do we have to make things better?" (O) and "What things might work against us to stop us achieving the opportunities?" (L).
  
• For each aspect to be evaluated, listed on the first column, the group identifies the strengths, weaknesses, opportunities and li-mitations, which are recorded in the relevant columns on the side.
  
• There are two ways to approach this exercise. You can go through all the strengths and then all the weaknesses for all the aspects to be evaluated; or you can go through the four categories for each item before moving onto the next item. A small test of the process use may help you decide which approach will work best in each instance.
  
• Where there are different opinions about an issue, the facilitator should help the group to reach a consensus. Some points may need to be discussed at length. Comments are recorded on the matrix only after agreement has been reached.

• the technique allows the different sides (positive and negative) of specific aspects to be identified and assessed for importance and therefore helps to set the basis for negotiations and trade-offs;
  
• it can be a means to build a consensus within a disparate group;
  
• SWOL encourages group creativity and helps to link the perception of how things are with the realistic expectations of how they could be, and to weigh the costs and benefits;

• a skilled facilitator is required for this process to be effective;
  
• sensitive subjects may arise. If this happens, the facilitator may choose to change the topic and return to the sensitive matter later;
  
• conflicting opinions can be difficult to accommodate, which may make some people hostile to the process;
  
• some individuals may try to dominate the discussion;
  
• summarizing discussions into short statements requires a facilitator with good listening and interpreting skills. Check with participants that they agree with the way views are recorded.

• radio programmes;
  
• natural group interviews;
  
• focus group interviews;
  
• semi-structured interviews with key informants;
  
• photo appraisal/slide language;
  
• observational walks and transect diagrams;
  
• trend analysis; and
  
• gender analysis

5.6 References and further readings on participatory tools and processes

<<< BACK

CONTENTS

NEXT >>>